Methods and systems for establishing and maintaining presence information of neighboring bluetooth devices

ABSTRACT

A communication device and method can include one or more processors operatively coupled to memory, a sensor and an output device, where the one or more processors to perform operations of identifying target person locations using internet searching and short range communication enabled devices such as Bluetooth LE devices.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claim priority to U.S. patentapplication Ser. No. 18/074,388, filed 2 Dec. 2022, which is acontinuation of and claims priority to U.S. patent application Ser. No.17/078,676, filed 23 Oct. 2020, which is a continuation of and claimspriority to U.S. patent application Ser. No. 16/556,534, filed 30 Aug.2019, which is a continuation application of and claims priority to U.S.patent application Ser. No. 14/507,784, filed 6 Oct. 2014, now U.S. Pat.No. 10,405,163, which is a utility patent application that claims thepriority benefit of Provisional Patent Application No. 61/887,437entitled “BLUETOOTH NETWORKING WITH PRESENCE DATABASE” filed on 6 Oct.2013, the entire contents of which are incorporated herein by referencein their entirety.

FIELD

The invention herein disclosed generally refers to methods and systemsfor networking Bluetooth LE enabled devices and creating a presencelist(s) and data base(s) reflecting the position of neighboring devicesin support of social networking, social media, private use cases,commercial business transactions and government use cases.

BACKGROUND

Whether at business related events (e.g. trade shows, meetings,conferences) or social gatherings (e.g. wedding receptions, partycelebrations, bars) or educational lectures situations often arise wherepeople meet for the first time or cannot remember the names ofparticipants they have met before or would like to have more backgroundinformation of their conversation partners or of the people around them.

At the same time, people are using social media to interact with otherpeople, and those media involve not only a communications path, but alsoan identity of the other participant and information about him/her whichis made available via the web or other interconnection medium. Inface-to-face interactions with other people it is often of interest toknow the identity and other information about the people engaging in theinterchange.

BRIEF DESCRIPTION OF THE FIGURES

The embodiments and the following detailed description thereof may beunderstood by reference to the following figures:

FIG. 1 depicts a scenario where a Bluetooth 4.0 or BT 4.0 or BluetoothLow Energy or BLE device [101] is scanning the environment to discoverother BLE devices in accordance with the embodiments;

FIG. 2 depicts a scenario where a BLE device [101] is requestingadditional data after discovery of neighboring devices in accordancewith the embodiments;

FIG. 3 depicts a scenario where a BLE device [101] advertises itspresence by broadcasting a universally unique identifier or UUID inaccordance with the embodiments;

FIGS. 4 a, 4 b, and 4 c depict the data layout for short and extendedreports in accordance with the embodiments;

FIG. 5 depicts the different modes that a participating BT 4.0 node mayassume in accordance with the embodiments;

FIG. 6 depicts a use case illustrating a business discovering anapproaching potential customer in accordance with the embodiments;

FIG. 7 depicts a use case where two people discover each other inphysical proximity through the use of BLE UUID in accordance with theembodiments;

FIG. 8 is a depicts a Voice-ID network and its functions in accordancewith an exemplary embodiment;

FIG. 9 depicts a method of Bluetooth 4.0 or WiFi triangulation tolocalize and track people who have been identified through biometricinput in accordance with an exemplary embodiment; and

FIG. 10 is an exemplary mobile device for use with authentication andlocalization in accordance with an exemplary embodiment.

All documents referenced herein are hereby incorporated by reference.

DETAILED DESCRIPTION

Digitally enabled social media is being used to share information underterms of engagement as parties see fit. This is made possible as all theparties operate in a virtual environment. Any party may choose toconnect on their terms and their time line. In other words, users whochoose to connect or engage, do so wherever and whenever they want. As aconsequence, social media has spawned a set of new barriers and threats.Is the focus now on communication quantity versus quality? Is there afocus on superficiality versus authenticity? In fact, social media hasthe potential to make us less social; a surrogate for the real thing.

As human beings, our only real method of connection is through authenticcommunication. Studies attributable to Professor Albert Mehrabian ofUCLA suggest that only 7% of communication is based on the written orverbal word and 93% is based on nonverbal language. This would mean that93% of face-to-face communication is conveyed through body languagewhich cannot be transmitted through the internet and can only partiallybe conveyed through the phone. Indeed, it's only when we can hear thetone of a voice or look into someone's eyes that we're able to know when“I'm fine” doesn't mean they're fine at all.

Contrast social engagement to traditional face-to-face settings, wherethe users are physically present. Actively engaging in social roleshelps people build self-esteem, physical wellness and a sense ofcommitment to the community around them. Furthermore, face-to-faceencounters provide for both verbal and body cues communicationsinvolving, facial expressions, hand gestures, gentle touch on theshoulder and eye contact. In addition, face-to-face communicationprovides for responses to be immediate, rather than dealing with thedelays from texts or e-mails and people are more likely to say whatcomes from the heart when they spend less time analyzing their response.

Face-to-face is often an ideal form of communications for these andother reasons but often requires a commitment of personal organizationto pre-determine a location and time to meet. In our busy society, theavailability of ones time is often in conflict with other obligations. Asolution would take advantage of spontaneously knowing where interestingor other interested people might be with respect to your proximity andto learn more about them through digital social media.

By definition, society would benefit from a technology that would bothleverage the efficiency of a computer-mediated communication platformbut focused on the intent of establishing and enhancing face-faceencounters; while expanding the opportunity to know of others in yourproximity and who may have a similar interest to you. The presentembodiments facilitate for the potential of improving both the qualityof quantity of face-to-face interactions by identifying existingcontacts and or potential new contacts who may be within a predeterminedphysical proximity.

Retail business could benefit tremendously from the knowledge andawareness of an approaching customer, what the customer is looking for,whether the customer has been at the business before and how often, thename and possibly a picture of the customer, and the ability toadvertise to the customer the availability and price of the merchandisethat is on their mind. Some of the embodiments herein intend to providesuch level of customer insight.

Some people visit bar type entertainment businesses to meet other peopleand to socialize. In case one's attention is drawn to a particularperson in that environment, it would be desirable to learn some detailsabout that person including the name and other personal details that arepublically available on social media sites or details that the targetperson has resident on his or her communication device and permitted tobe shared. Further, it may be desirable to obtain similar informationabout other patrons in a certain vicinity of one's communication device.Some of the embodiments herein intend to provide presence information insuch a surrounding and deliver this information via visual display on amobile communication device.

On college campuses, class sizes have become rather large. As a resultthe anonymity among students and teachers has grown to a point that ateacher may no longer be familiar with a particular student's name andperformance, whether the student is new or has attended the classbefore, or whether the student has a project completion due. Someembodiments herein can provide presence information, i.e. localizationalong with personal information and history information about presentstudents to aid the teacher in their interactions with the class. Inaddition, this data can be delivered via visual display on the teacher'smobile communication device or other computer system.

In a sports arena or venue, before or during a game it may be desirablelocating and communicating with friends and family or other people ofinterest. Further, after locating and identifying a friend, a familymember or a person of interest it may be desirable to contact thatperson using social media tools. This invention is intended to providepresence information about the surrounding stadium visitors includingtheir location along with personal information and displayed on a mobilecommunication device, and to provide the ability to make contact with aperson of interest using social media communications.

The use of social media sites promotes the establishment of virtualrelationships where each participant has control over their personalinformation that is disclosed on the associated websites. As a resultthe participants in a social media communication have a partial view ofpeople they are communicating with. People tend to form virtualcommunities comprising fellow members in fitness clubs, interest groups,churches, work environment, and similar. It may be desirable that incase members of such a virtual community are in a certain physicalproximity that their mobile devices provide an alert and offerinformation regarding an approaching fellow member or friend comprisinga name, a photo, and personal data, and possibly an acousticalsonification indicating a level of proximity and direction. Someembodiments herein provide this level of intelligence.

Bluetooth low energy, also known as Bluetooth 4.0 (BT 4.0), BLE,Bluetooth LE, or Bluetooth Smart, is a Bluetooth standard that isdesigned for ultra-low energy sensors requiring very little power. Someof the Bluetooth low energy devices that are already available include avariety of sports fitness sensors. In some embodiments, Bluetooth 4.0 isused in a meshed network configuration to connected personalcommunication devices and to exchange certain specific data relating tothe owner of the device. This data is uploaded to a cloud basedcomputing environment for forensic analysis and is used to complement avoice identification process and to achieve authentication quality, tocreate a presence database reflecting other BT 4.0 enabled devices inproximity of a discovering device, collecting personal data permittedfor sharing, and enable social media communication with the newlydetected devices. In addition, BT 4.0 RSSI signal strength is used tocompute the distance between two nodes and device triangulation is usedto aid in the localization of neighboring devices. Although BLE devicesare shown herein as an exemplary device, other short range communicationprotocols can be adapted to perform similar functions and theembodiments herein are not necessarily limited to BLE enabled devices.

U.S. patent application Ser. No. 14/475,556 describes Methods andSystems for a Voice Verification and Authentication Service,incorporated herein by reference. application Ser. No. 14/475,556describes assisting with identification of multiple speaking people inreal time situations. In particular, the method and systems disclosedmay provide for prequalification for Voice ID in conjunction withsingle/multi modality verification and validation systems by narrowingthe search space to increase the probability of a hit and the quality ofthe verification and validation results in terms of matchingprobability.

A variety of biometric sensors & detectors are used for security andaccess control management. Biometrics refers to methods for recognizingindividual people based on unique physical and behavioralcharacteristics. Many physical characteristics can be scanned by abiometric sensor which can scan for example, eyes, fingerprints, or DNAin order to verify a user and authenticate their identity. In thissense, biometric sensors may be able to benefit from a prequalificationprovided by the embodiments herein.

A wide range of mobile applications has become available to aid andpromote social media communications and interactions. One examplecomprises beacon-based applications that merchants use to detectapproaching customers, to learn about a customer's desire for aparticular product, or to advertise buying opportunities to a customer.A further example comprises mobile applications that entertainment andrestaurant businesses may use to provide premium service to theircustomers, or in a sport venue a mobile application may be downloaded toa visitor's mobile device to provide event related information andcommunications. In some embodiments, aggregation of the customerspecific information and data accumulated by the multitude of saidsocial applications can further provide for forensic analysis of thisdata for example in support of commercial business. Some embodimentsherein provide a social media aggregation engine and database that maybe used to obtain such detailed information about the behaviors of acustomer of interest.

The convergence of the social media interactivity and the ability ofspeaker identification/verification systems offers a unique capabilityto deliver talker specific information to the listener in manysituations. This information can be delivered via visual display, on acell phone, pad, or computer devices. Some embodiments herein can assistin the verification, authentication, and localization of the talker andother persons in proximity.

It is conceivable that some embodiments can be applied to environmentsand systems that comprise advanced glasses (such as Google Glass),earphones, headphones, watches, or acoustical transducers.

Methods, program products, and systems disclosed herein provide forsocial media networking solutions based on scanning and discoveringBluetooth LE (BLE) enabled devices and creating presence lists ofdiscovered devices and their user's personal data comprising:

-   -   1. A local presence list residing on a BLE scanning device        reflecting discovered devices and personal user data obtained        from devices in BLE communication range.    -   2. A local presence list where devices on the list are further        categorized into nearby devices and present devices based on the        Received Signal Strength Indicator (RSSI) value.    -   3. A central user presence database populated by BLE discovered        users comprising an accumulative list of present users across a        multitude of devices as well as entries for previously known        users that have been identified in the past but are no longer on        any device's presence list

The mobile instrument can store all the discovered devices without thenecessity to communicate with a cloud-based database, such system beingself-organizing.

-   -   4. In an embodiment the central user database may be accessed        through cloud-based communications and may be used to obtain        present user information as well as historical user information,        for example but not limited to:        -   a. Device specific data        -   b. User Name        -   c. User Picture        -   d. Company/Social Media        -   e. Contact phone        -   f. Contact Email        -   g. Location

In an embodiment, the central database may be functioning as a socialmedia aggregation engine and database, gathering data about individualsfrom social media applications as used for example, but not limited to,in retail business, entertainment and/or sports events where they areused to provide customers with a premium service and communications,where the gathered data may be analyzed using forensic software programsand the results may be offered to, for example, commercial customers tomake better-informed advertisements and decisions when serving theirclientele.

In an embodiment the central social media aggregation database may beused to keep track of where a person goes, and how much time anindividual spends in a monitored location and provide this data in aform of a histogram showing in chronological order the time-locationtrace of an individual.

Further in an embodiment, the social media aggregation database may beused for on-demand interrogation of individuals in order to learn abouttheir whereabouts, where they spent their time, where they are currentlylocated, and other similar information.

In an embodiment, the social media aggregation database can be used toidentify a multitude of individuals (swarm) who display similar or samebehavioral patterns for example in a shopping mall and provideprogressive information of their movements.

In an embodiment, the social media aggregation database system may useweb crawling techniques to obtain social media data from a multitude ofsocial media sites.

Further in an embodiment of the invention, when customers download andinstall social media applications, a cookie can be installed on thecustomer's communication device to prompt for permission for theapplication to upload social and behavioral data to the social mediaaggregation data base.

In an embodiment the central user database can be accessed throughcloud-based communications and the data may be used to obtain presentuser information and behavior as well as historical user information andbehavior, which may be used by forensic analysis programs to derivebehavior profiles of users.

In an embodiment, a software program component (SPC) is installed on BLEdevices to support the scanning and data exchange process throughbroadcasting and responding to an application-specific BLE type UUID,comprising:

-   -   a. A Candidate Presence List (CPL) software program to create,        maintain, and update presence lists;    -   b. A Configuration Application Program Interface (API) logically        coupled with the CPL program facilitating provisioning of static        data for advertising and other system parameters, and setting a        user's profile information for sharing upon request from other        devices;    -   c. A mode change API logically coupled with the CPL program        facilitating controlling the mode of operation, which can be        scanning for candidates or advertising (broadcasting) to be        discovered; and    -   d. CPL output API providing a list of present devices with all        required ID information used for further processing.

In one embodiment the SPC application installed on a BLE enabled devicemay recognize SPC enabled BLE devices in communication range and alertthe user with visual and textual displays comprising the name andpicture of users of the detected devices and it can issue an acousticalsonification indicating an approximate distance to a discovered device.

In an embodiment, a BLE device alternately enters a broadcasting modeand a listening mode to survey surroundings, create a presence list, andrequest more data from discovered neighbors.

In an embodiment, a BLE UUID receiving device evaluates the RSSIindicator and calculates the distance to the broadcasting device basedon the formula RSSI[dbm]=−(10n log 10(d)−A) where d is the distance andA is the offset which is the measured RSSI 1 meter point away from theBLE device.

Further in an embodiment, a calculated distance to a broadcasting deviceis used to classify a corresponding entry in the presence list as“nearby” if the measured value is less than a specified threshold.

In an embodiment, BLE based device triangulation in combination with anRSSI evaluation may be used to determine the position of a BLE device.In an embodiment, the system can be set to discover any number ofparticipants within a specific area. Furthermore, the system can informthe user of approaching users.

In an embodiment, some embodiments can be applied to a VoiceIdentification and Authentication Service as described in U.S.Non-Provisional application Ser. No. 14/475,556 to assist withidentification of multiple speaking people in a real time situation. Themethod and systems disclosed therein can provide for prequalificationfor Voice ID in conjunction with single/multi modality verification andvalidation systems by narrowing the search space to increase theprobability of a hit and the quality of the verification and validationresults in terms of matching probability.

Upon establishing a presence list comprising a multitude of biometricand personal data, the application software can control the device toobtain voice signatures for each candidate on the nearby presence listby accessing a remote server using a TCP/IP connection specifying anoptimized search field based on the data obtained through BLE UUID basedcommunications.

The features of the embodiments, which are believed to be novel, are setforth with particularity in the appended claims. The embodiments maybest be understood by reference to the description, taken in conjunctionwith the accompanying drawings

While the specification concludes with the claims defining the featuresof the embodiments that are regarded as novel, it is believed that theembodiments may be better understood from a consideration of thefollowing description in conjunction with the drawings figures, in whichlike reference numerals are carried forward.

The terms and phrases used herein are not intended to be limiting butrather to provide an understandable description of the embodiments.

The terms “a” or “an”, as used herein, are defied as one or more thanone.

The term “another”, as used herein, is defined as at least a second ormore. The terms “including” and/or “having” as used herein, are definedas comprising (i.e. open transition). The term “coupled” or “operativelycoupled” as used herein, is defined as connected, although notnecessarily directly, and not necessarily mechanically.

Architecture and Operation

FIG. 1 illustrates a realization of a Bluetooth 4.0 LE meshed ad-hocnetwork according to an embodiment of the present invention. The diagramdepicted belongs to a series of figures describing system architecturein relevance to the software application's operation and identificationof potential speakers in the vicinity of a user's device. User device(101) has started its BLE scan and is listening for UUID broadcasts fromother devices. In the example shown user devices 102, 103, and 104 arebroadcasting their device identifications in form of a UUID. The datapackage transported via in the UUID is referred to as a “Short Report”data package 401 which can include a device address, an address type, adevice user ID, an advertised transmit level, and an RSSI. FIG. 4 aprovides an example of a short report. Based on received advertisinginformation (UUID) from the broadcasting devices, device 101 updates itsdevice presence list to include discovered devices and to remove devicesno longer found during the last scan. The latter information is used tocreate a disappeared device list which includes nearby devices andpresent devices and which may be reported upon demand.

Further to FIG. 1 , device 101 evaluates Advertised TX power and RSSIand calculates a distance to the broadcasting devices. The result iscompared with a pre-specified distance value. If the calculated distanceis equal or less than the pre-defined value, the broadcasting device isassigned the attribute “nearby device” which is recorded in the presencelist. If the calculated distance is greater than the predefined distancevalue, then the device is categorized as “present device”, which isrecorded in the presence list.

Further to FIG. 1 , based on received advertising information from thebroadcasting devices, device 101 updates its device presence list byvalidating what listed existing “nearby devices” are maintaining theirattribute based on proximity criteria (Advertised TX power vs. ReceivedRSSI Level), otherwise their attribute is changed to “present device”.

Further to FIG. 1 , based on received advertising information from thebroadcasting devices, device 101 updates its device presence list byvalidating what listed existing “present devices” meet the proximitycriteria (Adv TX power vs. Received RSSI Level); in that case, theirattribute in the presence list is changed to “nearby device”.

Further to FIG. 1 , based on received advertising information frombroadcasting devices, device 101 updates its device presence list to addnew devices discovered during the last scan and to assign the “nearbydevice” attribute to devices that meet the proximity criteria(Advertised TX power vs. Received RSSI Level); all other new devices bydefault have the “present device” attribute in the presence list.

Further to FIG. 1 , device 101 may create a list of known nearby deviceswith information extracted from the corresponding advertising payloadonly (short report) and report it upon demand.

FIG. 2 continues to illustrate a realization of a Bluetooth 4.0 LEmeshed ad-hoc network as depicted in FIG. 1 . FIG. 2 illustrates phase 2where a list of close neighbors is created and where more information isrequest from them. The diagram depicts a scenario, where device 101after completing its discovery cycle of other BLE neighboring nodes mayrequest additional information from the nodes identified on its “nearbydevice” list. For example, device 101 may ask device 102 “I'm John” anddevice 104 “I'm Dave” for more information, whereas device 103 “I'mMary” has been determined to be to far from device 101 and thus is notplaced upon the “nearby device” list. This request may be initiated ondemand. The resulting report is referred to as “extended report”. Anexemplary structure is provided in FIG. 4 b with the InformationContainer Structure 402 and in FIG. 4 c with the Extended Report datasample 403.

Further to FIG. 2 , upon demand for an extended report on a specifiednearby device, the device 101 first checks if a required extended reportalready exists as a result of a previous enquiry. If this data is notavailable, device 101 may establish a connection with the specifiednearby device and extract information for the extended report. Uponreceiving the data the connection is closed, the information associatedwith this device is stored, and an extended report may be provided inresponse to the request.

FIG. 3 continues to illustrate a realization of a Bluetooth 4.0 LEmeshed ad-hoc network as depicted in FIG. 1 . FIG. 3 illustrates a phase3 where neighboring devices enter a discovery operation. The diagramdepicts a scenario, where device 101 switches upon demand into anadvertising mode and broadcasts application-specific UUID and associatedadvertising data so it can be discovered by scanning BLE devices incommunication range. Device 101 reports when the advertising mode hastimed out or has been canceled on demand. In advertising mode, a BLEdevice supports BLE connectivity requests to provide extended reportdata. For example device 101 “I'm Mike” can advertise it's presence toother devices (e.g., device 102 “I'm John”, device 103 “I'm Mary”,device 104 “I'm Dave”) which can complete their own discovery of otherdevices.

FIG. 4 a provides an exemplary short report format. Short reportinformation is limited by the amount of data that can be integrated inthe BLE advertising package.

FIG. 4 b provides a generic structure for an information containerstructure 402 that may be appended to a short report to arrive at anextended report. An information container is a version controlled datastructure. FIG. 4 c provides an exemplary embodiment of an extendedreport 403.

FIG. 5 depicts the various possible states a BLE device can assume andtheir transitions under control of a candidate presence list (CPL)application software. For example when the device is initially in “idle”it can start a scan 503 and receive a short report from the scan. Whilein “idle” the device can get extended report data 504 and receive theextended report data. While in “idle” the device can start advertising502 and can determine when advertising is done, and can additionallyrequest extended reports 501.

In one embodiment the SPC software program installed on a first BLEenabled device may recognize a second SPC enabled BLE devices incommunication range and alert the user of the first device with visualand textual displays comprising at least the name and picture of theuser of the detected device and it may issue an acoustical sonificationindicating an approximate distance to the discovered device.

In one embodiment the SPC software program installed on a first BLEenabled device may recognize a multitude of SPC enabled BLE devices incommunication range and alert the user of the discovered devices withvisual and textual displays comprising at least the names and picturesof the users of the detected devices.

In one embodiment the SPC software program may report events ofdiscovering neighboring devices and their exchanged data to a CentralUser Database.

In one embodiment a Central User Database is provided access throughInternet cloud based communications that receives and records presencedata from the local SPC application software instantiations for analysisand sharing purposes.

In an embodiment, the central database may be functioning as a socialmedia aggregation engine and database, gathering social media data aboutindividuals from social media applications as used for example, but notlimited to, in retail business, entertainment and/or sports events wherethey are used to provide customers with a premium service andcommunications.

FIG. 6 depicts a use case in a shopping mall scenario. The illustrationcomprises:

-   -   a. A shopping mall 600 where a business B is scanning for BLE        customers such as customer “Ralph” who owns device 601;    -   b. Customer Ralph and his device 601 is approaching business B;    -   c. A central user database 608 containing presence data for BLE        devices that have downloaded and installed the SPC App 605;    -   d. The additional Social Media Aggregator DB function 609 of the        central user DB 608;    -   e. A DB engine 607 that controls access to and storage logic for        the central databases; and    -   f. An Internet Communication Network (Cloud) 604.

In an embodiment customer device 601 and business B are interfacing thecentral User database 608 to upload presence information, social mediadata, location, time and event information, while customer device 601 isapproaching business B. Further in the embodiment business B discoversthe UUID of customer device 601 when customer device 601 comes withinthe BLE communication range of business B, and directs an advertisement602 to customer device 601 soliciting download of a business Bcustomized web application to conduct a sale.

FIG. 7 depicts a use case in a shopping mall 700 scenario where twopeople (Ralph and Mary via their mobile devices respectively) (701,702)who may have had to this point a virtual relationship over social mediacommunications, meet physically in the mall alerted by their SPCApplication 705 upon discovery of a UUID. Through interfacing thecentral user database 708 via communication links 703, cloudcommunications network 704, and database engine 707 they (Ralph and Maryvia their devices 701 and 702) may quickly retrieve additionalinformation about each other presented on their mobile devices 701 and702.

Further in an embodiment, the social media aggregation database engine607 or 707 may employ forensic software programs 710 to aggregate andanalyze data received from social media applications.

In an embodiment the central social media aggregation database engine607 or 707 may be used to keep track of where a person goes, and howmuch time an individual spends in a particular location, and providethis data in form of a histogram showing in chronological order thetime-location trace of an individual.

In an embodiment the central social media aggregation database engine607 or 707 may be used by commercial social media applications to obtaintracking data and personal data of customers in BLE proximity to providedirect advertisements to the customer based on parameters comprisingtime, tracking and buying history, and UUID exchanged data revealing thecustomer's desire for certain products.

Further in an embodiment, the social media aggregation database engine607 or 707 may be used for on-demand query of individuals comprisingtheir current location, previous locations, and time spent in previouslocations.

In an embodiment, the social media aggregation database 609 or 709 maybe used to identify a multitude of individuals (swarm) who displaysimilar or same behavioral patterns, for example in a shopping mall, andto provide progressive information of their (the swarm's) movements.

In an embodiment, the social media aggregation database engine 607 or707 may use web crawling techniques to obtain social media data from amultitude of social media sites.

Further in an embodiment, when customers download and install socialmedia applications, a cookie may be installed on the customer'scommunication device to prompt for permission for the application toupload social and behavioral data to a social media aggregation database such as databases 609 or 709.

Those with ordinary skill in the art may appreciate that the elements inthe figures are illustrated for simplicity and clarity and are notnecessarily drawn to scale. For example, the dimensions of some of theelements in the figures may be exaggerated, relative to other elements,in order to improve the understanding of the present invention.

In some embodiments, described technology may be applied to environmentsand devices comprising transducers or earphones or wristwatches of thefollowing categories:

Transducers:

-   -   Bone conduction, Transdermal conduction, Magnetic transducers,        Optical transducers, such as Fiber-Optic Acoustic Sensors        (FOAS), Electromagnetic Acoustic Transducer (EMAT),    -   Piezoelectric Transducers (can make contact or be used in a        non-contact enablement) Omni-directional and directional        transducers,    -   Earphones:        -   Canal phone        -   Headphone        -   Earphone        -   Mono Acoustic Earphone        -   Stereo Acoustic Earphone        -   Binaural Earphone Hearing aid        -   PSAP (assertive listening devices)        -   Glasses (such as Google Glass)        -   In Canal Earphone        -   In Concha Earphone        -   Behind The Ear (BTE devices) Earphone        -   Active Glasses (such as Google Glass)        -   Heads up display    -   An electrode array, which is a group of electrodes that collects        the impulses a the stimulator and sends them to different        regions of the auditory nerve.        -   Wrist watches        -   Time Pieces        -   Heart Monitors        -   Fitness watches        -   Wrist wrapped monitoring devices

In some embodiments, a Voice Identification Service as described in U.S.Non-Provisional application Ser. No. 14/475,556 can be used to assistwith identification of multiple speaking people in a real timesituation.

For example with reference to FIG. 8 , in some embodiments, FIG. 8illustrates a realization of a Voice-ID network system 800 according toan embodiment of the present embodiments. As shown in FIG. 8 , thecomponents of the Voice-ID network include:

-   -   a voice identifier database 801 that contains the Voice IDs of        opted in Voice-ID network subscribers;    -   a database server 802 that controls the opt-in procedure;    -   a communications network 810;    -   a voice ID App 803A that is installed on a mobile device 803 to        deliver visual profile information and persona authentication in        the vicinity of the device and controls the recording and        delivery of voice samples to create voice prints;    -   a social media service (e.g., social networking sites or        applications) 806 to communicate with other people that in        conjunction with the voice identifier database 801 identifies        other participants made available via the web or other        interconnection means;    -   An ID Verification Service System 804 that provides an interface        to a multitude of users in need of ID verification 805; and    -   At least one local mobile device 807 having a microphone and        providing an established social media connection (e.g., friend)        to the voice ID App 803A.

The Voice IDs stored in the voice identifier database 801 are alsoreferred to as voice prints or voiceprints. The voiceprint is anefficient distillation of the speech of a talker which containsinformation about his or her identity, but which does not necessarilycontain information about the content of the speech used to make thevoiceprint. The voice prints include 1) a voice sample, for example, aspoken utterance, or phrases, digitally captured and stored on media, 2)a profile associated with the voice sample (described below), andoptionally 3) device specific information, for example, a globalpositioning system (GPS) location, a Wi-Fi network, a MAC address, an IPaddress, and a unique device identifier, such as provided by the mobiledevice running the voice ID App 803A.

The profile can include information such as a name, email address,identification number, IP address, vehicle registration plate number,driver's license number, biometric data, face, iris print, fingerprints,or handwriting, credit card numbers, digital identity, date of birth,birthplace, genetic information, telephone number, login name, screenname, nickname, or handle, country, state, or city of residence, age,gender, race, name of a school attended, name of workplace, location ofworkplace, grades, salary, or job position, or criminal record. Theaforementioned list of profile information is not meant to be exhaustiveand only provides a sample of the types of information that can be usedin the embodiments herein. As discussed later, such profile informationcan be used to corroborate an identity.

In the modern collection of algorithms for analyzing audio and speechsignals, one aspect of the embodiments is a method to identify thetalker of any particular utterance, independent of the informationcontained in the transcription of that utterance. That is, the talker isidentified on only the characteristics of that person's voice, and noton the content or phrase of a spoken utterance. Also, in face-to-faceinteractions with other people the method described herein identifiesinformation about the people whose voice are captured and/or engaging ina spoken interchange (There might be a mix of people, for instance in aconference room which includes people on a speaker phone, or in aconference setting with some people at a remote site). The Voice-IDnetwork system of FIG. 8 obtains in near real-time the ID, profile andpersonal information of individuals merely by capturing their voiceprints. This greatly enhances the ability for people to socialize andinteract in a variety of situations.

Another aspect of the embodiments is the building and updating of alarge and secure voice identifier database 801 of voice prints throughthe utilization of social media, and further incorporating the voiceprints to enable enhanced social media value and security. The Voice IDdatabase query system consists of a database that contains secured andencrypted voice ID and other profile data for opted-in individuals, anacquisition method for such voice ID and other profile data forindividuals through the use of social networking capabilities, an IDverification system that is used both in social networking as well ascommercial transactions to deliver the ID of humans based on their voicesamples, and a mobile application (referred to as VIDA in the following)to facilitate the recording and translation of voice samples into voiceID data and profile data for the individual recorded, and which providesa visual representation of the data obtained. Once configured, thisdatabase allows for a novel ID verification service based on capturingvoice samples that benefits a multitude of users ranging from financialinstitutions to the control of set top boxes and vehicle access, to namea few. The large scale building up of the voice ID entries and the voiceidentifier database 801 is further achieved through the exploitation ofcontact and friends lists that are part of social networking or socialmedia. Voice identification is performed through a profile searching ofsocial media. This efficient searching identifies voice samples ofunknown talkers through access of contact or friends lists of a deviceused to capture the voice samples associated with a user of the deviceby way of the voice print.

The convergence of the social media interactivity and the ability ofspeaker identification/verification systems offers a unique capabilityto deliver talker specific information to the listener in manysituations. This information can be delivered via visual display, on acell phone, pad, or computer devices. While Voice ID deliversidentification of people with a high degree of probability it may becomedesirable in certain situations to have certainty in the speaker'sidentification to a point that the person's voice can be authenticated.For example, in a large meeting room it may be helpful to associate thevoice ID of a speaker with the speaker's location in the room to clearlydelineate who the ID'ed voice belongs to. In another situation, at thecheckout point of a retail store it may be desirable that a customer canauthenticate himself or herself just by delivering a voice sample, andthereby complete the payment of the merchandise. Or in a meeting room,when a person approaches another participant the device display canreact and adjust social display properties to focus on the approachingperson.

The ability to localize other people using Bluetooth 4.0 connectivitydata enhances the Voice ID process significantly by increasing matchingprobabilities and allowing for authentication. These embodiments areleveraging the capabilities of a Bluetooth 4.0 system to narrow down thesearch space and to accomplish an accuracy of Voice Identificationsuitable for authentication. Bluetooth low energy, also known asBluetooth 4.0 (BT 4.0), BLE, Bluetooth LE, and Bluetooth Smart, is aBluetooth standard that is designed for ultra-low energy sensorsrequiring very little power. Some of the Bluetooth low energy devicesthat are already available include a variety of sports fitness sensors.In some embodiments, Bluetooth 4.0 is used in a meshed networkconfiguration to connected personal communication devices and toexchange certain specific data relating to the owner of the device. Thisdata is used to complement the voice identification process and toachieve authentication quality. In addition, BT 4.0 is used to computethe distance between two nodes based on signal quality (RSSI).

FIG. 9 depicts a method of Bluetooth 4.0 or WiFi triangulation tolocalize and track people who have been identified through theirbiometric input such as a voiceprint. Recall, one objective achievedherein by way of the method and system disclosed is the interwork ofBluetooth 4.0 connection functions and data with Voice ID verificationfor persona authentication and localization. For instance, is the personwhom the phone is capturing live voice samples from, an individual whichthe user of the phone knows or is able to infer from social media andestablished social connections. Aside from social media connections, themethod and system herein includes location information to verify orcorroborate the biometric input and accordingly the authentication ofthat person's identify. For example, upon capturing a voice sample, thedevice checks on a friend list and on social media to assess whether afriend is scheduled to be at an event. The device, or the server, thennarrows down the searching of a voice print match to the narrowedpresence list identifying which individuals may be present at thelocation.

For assessing a person's location, the VIDA mobile application 103A inone configuration in a mobile device 809 in system 900, measures the BT4.0 received signal strength and computes an estimate of the distance tothe UUID broadcasting BT 4.0 device. By having at least three VIDAenabled devices in 802.11 WiFi or BT 4.0 I-hop proximity, the VIDAapplication identifies and tracks the location and movement of otherdevices by means of triangulation. The accuracy of the localizationincreases with the number of VIDA enabled devices in the proximity. Theoutcome of the distance/localization calculation is reflected and shownon the presence screens of the devices by centering the display on thedevice user who may be closest in proximity. As illustrated in FIG. 9 ,this is achieved from a signal strength measurement to each of themobile devices (see A, Y, Z), querying the voice ID database 801 for apotential location based on a presence profile maintained in thepresence server 805 and the persona of the individual from social media,and comparing the location with the potential location to validate theauthentication of the persona of the individual from the voice sample.The VIDA enabled devices 809 may be equipped with a GPS system (see FIG.10 ; location unit 358). The on-board device GPS system may be usedfacilitating the activation and deactivation of the device BT 4.0 and802.11 WiFi radios upon entering and leaving a building. For example,when entering a building the device 809 may be made aware of this eventby its GPS system and may turn on its 802.11 or Bluetooth 4.0 radios.Upon leaving a building the device may turn off its 802.11 or Bluetooth4.0 radios upon notification by its GPS system indicatingre-synchronization with GPS satellites. The decision to turn on or offthe 802.11 or Bluetooth 4.0 radios may be overridden by the device user.

A proximity analysis can be conducted to the local devices from thesignal strengths received, which can include refining the presence listto include only devices that are in a specified distance range fromcorresponding distance estimates in view of the proximity analysis basedon a voice quality metric, and capturing voice samples from the devicesidentified in the presence list for ensuring voice quality verificationand authentication. The VIDA mobile application 103A conducts theproximity analysis by using the computed distance to other devices,derived from the BT 4.0 received signal strength indicator, and limitits presence list to those devices that are in a specified distancerange. This may allow limiting the presence list to those users whosevoices can be recorded with sufficient quality for Voice ID verificationand authentication. The VIDA application 103A may continually andautomatically record the surrounding audio field and upload therecording to a central database according to proximity and location.

Also, a time stamp is marked for each one or more voice samples capturedfrom the devices, and the one or more voice samples can be time orderedaccording to the time stamp. This can be done by the devicesindividually or at the server which has group ID and time stampinformation available. A speech to text conversion of the one or morevoice samples is then performed according to the time ordering. Thisallows all participants in the area to receive a text of theconversation, with indication of who spoke (voice sample information)and when (a time stamp). In one arrangement, the VIDA mobile application103A uses a probabilistic anti-model for the purpose of VoiceIdentification in the processing of voice recordings for speech to textconversion. The probabilistic anti-model for a particular device maybeset up in such a way that the voice of the device owner is ID'ed andtime stamped as opposed to all other surrounding voices and audio.Applying an user specific probabilistic anti-model to each device inproximity of a device with the device user being the ID'ed and timestamped voice, then the resulting recordings of all devices that areparticipating in a conversation may be converted to a text fileprecisely reflecting all parties of a conversation with their timestamps and conversation text. Further, the recorded conversations ofsaid users may be reconstituted to a single audio file composed of theidentified and dominant voice segments of each user's recording.

In one arrangement, the VIDA enabled device 103A may solicit and recruitother non-registered devices to join the Voice ID verification networkand download and execute the VIDA application. The recruiting includesestablishing a Bluetooth connection with the unknown device broadcastingand receiving a BT 4.0 UUID with the Bluetooth MAC address of theunknown device, Interrogating cloud based database services and socialmedia sites to find and match the UUID or BT 4.0 MAC address to a mobilephone number associated with the unknown device, prompting the unknowndevice to join the Voice ID network via a social media invocation suchas a text message or similar including a link to the VIDA application,and using social media to establish a dialog with the user of theunknown device

In an exemplary scenario the VIDA enabled devices may establish aconnection with the unknown device using BT 4.0 UUID broadcasting andscanning. The received UUID of the unknown device may contain theBluetooth address of that device. The VIDA application 103A may use thereceived UUID and Bluetooth 4.0 MAC address to search cloud baseddatabase services and social networking sites to match the UUID/MACaddress to a mobile phone number. This may require that the user of theunknown device has previously registered his/her phone number with anonline service or social media site. Once the search for a phone numberis successful the VIDA application may prompt the unknown device to jointhe network via social media invocation such as text messaging orsimilar that includes a link to the VIDA application. Agreement toopt-in to the Voice ID network may be given via a special confirmationcode or by accepting the download of the VIDA application.

FIG. 10 depicts various components of a multimedia device 809 suitablefor use for use with, and/or practicing the aspects of the inventiveelements disclosed herein, though is not limited to only the methods orcomponents shown or described. As illustrated, the device 809 comprisesa wired and/or wireless transceiver 352, a user interface (UI) display354, a memory 356, a location unit 358, and a processor 360 for managingoperations thereof. The media device 300 can be any intelligentprocessing platform with Digital signal processing capabilities,application processor, data storage, display, input modality liketouch-screen or keypad, microphones, speaker 366, Bluetooth, andconnection to the internet via WAN, Wi-Fi, Ethernet or USB. Thisembodies custom hardware devices, Smartphone, cell phone, mobile device,iPad and iPod like devices, a laptop, a notebook, a tablet, or any othertype of portable and mobile communication device. Other devices orsystems such as a desktop, automobile electronic dash board,computational monitor, or communications control equipment is alsoherein contemplated for implementing the methods herein described. Apower supply 362 provides energy for electronic components.

In one embodiment where the media device 809 operates in a landlineenvironment, the transceiver 352 can utilize common wire-line accesstechnology to support POTS or VoIP services. In a wirelesscommunications setting, the transceiver 352 can utilize commontechnologies to support singly or in combination any number of wirelessaccess technologies including without limitation Bluetooth™, WirelessFidelity (WiFi), Worldwide Interoperability for Microwave Access(WiMAX), Ultra Wide Band (UWB), software defined radio (SDR), andcellular access technologies such as CDMA-1X, W-CDMA/HSDPA, GSM/GPRS,EDGE, TDMA/EDGE, and EVDO. SDR can be utilized for accessing a public orprivate communication spectrum according to any number of communicationprotocols that can be dynamically downloaded over-the-air to thecommunication device. It should be noted also that next generationwireless access technologies can be applied to the present disclosure.

The power supply 362 can utilize common power management technologiessuch as power from USB, replaceable batteries, supply regulationtechnologies, and charging system technologies for supplying energy tothe components of the communication device and to facilitate portableapplications. In stationary applications, the power supply 362 can bemodified so as to extract energy from a common wall outlet and therebysupply DC power to the components of the communication device 300.

The location unit 358 can utilize common technology such as a GPS(Global Positioning System) receiver that can intercept satellitesignals and there from determine a location fix of the portable device809. The controller processor 360 can utilize computing technologiessuch as a microprocessor and/or digital signal processor (DSP) withassociated storage memory such a Flash, ROM, RAM, SRAM, DRAM or otherlike technologies for controlling operations of the aforementionedcomponents of the communication device. The device 809 can furtherinclude at least one sensor 364 such as a microphone to capture a voicesample, a camera to capture an image, an iris scanner to capture an irisscan, or a fingerprint scanning device to capture a fingerprint. Otherbiometric inputs are contemplated and are not limited to those shown ordescribed.

In some embodiments, the SPC software program installed on a first BLEenabled device may recognize a multitude of SPC enabled BLE devices incommunication range and apply this information to narrow down the searchspace for a subsequent Voice identification and authentication, therebyincreasing the probability of a match.

In an embodiment, the SPC software program installed on a first BLEenabled device may apply UUID obtained data from a second BLE enableddevice to assist single or multi modality verification systems, therebyincreasing the probability of a match.

Upon establishing a presence list the SPC application software maycontrol the device to obtain voice signatures for each candidate on thenearby presence list by accessing a remote server using a TCP/IPconnection.

It will be appreciated that the various steps identified and describedabove may be varied, and that the order of steps may be adapted toparticular applications of the techniques disclosed herein. All suchvariations and modifications are intended to fall within the scope ofthis disclosure. As such, the depiction and/or description of an orderfor various steps should not be understood to require a particular orderof execution for those steps, unless required by a particularapplication, or explicitly stated or otherwise clear from the context.

While the invention has been disclosed in connection with the preferredembodiments shown and described in detail, various modifications andimprovements thereon will become readily apparent to those skilled inthe art. Accordingly, the spirit and scope of the present invention isnot to be limited by the foregoing examples, but is to be understood inthe broadest sense allowable by law.

Those with ordinary skill in the art may appreciate that the elements inthe figures are illustrated for simplicity and clarity and are notnecessarily drawn to scale. For example, the dimensions of some of theelements in the figures may be exaggerated, relative to other elements,in order to improve the understanding of the present embodiments.

All documents referenced herein are hereby incorporated by reference.

What is claimed is:
 1. A device, comprising: a user interface; amicrophone; a memory configured to store instructions; and a circuitcoupled to the memory, the microphone, and the user interface, whereinthe circuit performs operations by executing the instructions, theoperations defined by at least the following acts comprising: recordinga voice signal of a non-user of the device; generating a voice samplefrom the voice signal; identifying the non-user by comparing the voicesample to stored voice samples in a database; retrieving presentinformation of the non-user from a first database; retrieving historicalinformation of the non-user, wherein the historical information isretrieved from at least the first database or a second database or acombination of both; sending a portion of the present information to theuser interface; and sending a second portion of the historicalinformation to the user interface.
 2. The device according to claim 1,wherein the historical data includes data from the internet including atleast one of social media posts, publications, recordings, postings infitness club's social media, posting in interest group social media,posting in church social media, posting in work environment socialmedia, posting in video social media, a photo, acoustical data, or acombination thereof.
 3. The device according to claim 1, wherein thepresent information includes at least one of a name, email address,identification number, IP address, vehicle registration plate number,driver's license number, biometric data, face, iris print, fingerprints,or handwriting, credit card numbers, digital identity, date of birth,birthplace, genetic information, telephone number, login name, screenname, nickname, or handle, country, state, or city of residence, age,gender, race, name of a school attended, name of workplace, location ofworkplace, grades, salary, or job position, or criminal record or acombination thereof.
 4. The device according to claim 1, wherein theacts further comprise: updating the second database with historicalinformation, wherein the historical information is social media postsfrom the non-user extracted from the internet.
 5. The device accordingto claim 4, wherein the acts further comprise: updating the seconddatabase with multiple voice samples from the identified social mediaposts.
 6. The device according to claim 5, wherein the acts furthercomprise: identifying the locations of the social media posts.
 7. Thedevice according to claim 6, wherein the acts further comprise: storingthe locations in the second database.
 8. The device according to claim1, wherein the user interface is a touch screen display.
 9. A method,comprising: recording a voice signal of a target person; generating avoice sample from the voice signal; identifying the target person bycomparing the voice sample to stored voice samples in a database;retrieving present information of the target person from a firstdatabase; retrieving historical information of the target person,wherein the historical information is retrieved from at least the firstdatabase or a second database or a combination of both; sending aportion of the present information to the user interface; and sending asecond portion of the historical information to the user interface. 10.The method according to claim 9, wherein the historical data includesdata from the internet including at least one of social media posts,publications, recordings, postings in fitness club's social media,posting in interest group social media, posting in church social media,posting in work environment social media, posting in video social media,a photo, acoustical data, or a combination thereof.
 11. The methodaccording to claim 9, wherein the present information includes at leastone of a name, email address, identification number, IP address, vehicleregistration plate number, driver's license number, biometric data,face, iris print, fingerprints, or handwriting, credit card numbers,digital identity, date of birth, birthplace, genetic information,telephone number, login name, screen name, nickname, or handle, country,state, or city of residence, age, gender, race, name of a schoolattended, name of workplace, location of workplace, grades, salary, orjob position, or criminal record or a combination thereof.
 12. Themethod according to claim 9 further comprising: updating the seconddatabase with historical information, wherein the historical informationis social media posts from the non-user extracted from the internet. 13.The method according to claim 12 further comprising: updating the seconddatabase with multiple voice samples from the identified social mediaposts.
 14. The method according to claim 13 further comprising:identifying the locations of the social media posts.
 15. The methodaccording to claim 14 further comprising: storing the locations in thesecond database.
 16. The method according to claim 9, wherein the userinterface is a touch screen display.